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STRONGER  TESTS  USING  HETEROSCEDASTIC  ANOVA 
WITH  A  SIMULATION  OF  MULTIPLE  OBJECTIVE  BUDGETING 
CASE  STUDY 


The  traditional  F-tests  assume  that  all  the  population  variables 
under  consideration  have  unknown  means  with  equal  unknown  variances.  The 
experimenter  can  control  Type  I  error  but  not  Type  II  error  under  the 
traditional  F-test  simulation  experiment.^ 

The  objective  of  this  note  is  to  show  that  the  problem  of  controlling 
Type  II  error  can  be  solved  by  using  a  new  F-test,  which  also  allows  for 
unequal  unknown  population  variances.  The  traditional  analysis  of  vari¬ 
ance  (ANOVA)  is  based  on  the  assumptions  of  normality,  independence  of  the 
statistical  errors,  and  equality  of  the  variances  of  the  errors.  Studies 
of  the  robustness  of  the  F-test  have  shown  that  the  violation  of  normality 
has  little  effect  on  inferences  about  the  means.  However,  the  violation 
of  independence  or  equality  of  variances  can  have  a  serious  effect  on 
inferences  about  the  means,  especially  if  the  cell  sample  sizes  are 
unequal  (see,  for  example,  Scheffe  [1959]  or  Bishop  [1976]).  In  practice, 
the  assumption  of  equality  of  error  variances  seems  to  be  often 
unjustified;  in  fact,  even  when  Lite  error  variances  are  equal,  the  power 
of  the  F-test  depends  upon  the  unknown  common  variance,  which  renders  it 
difficult  to  plan  an  experiment  rationally.  Recently,  Bishop  and  Dudewicz 
[  1978]  £  198 1 ]  developed  new  ANOVA  procedures  in  the  contexts  of  the  one¬ 
way  layout  and  higher-way  layouts.  Their  procedures  allow  unequal  and 
unknown  population  variances  and  give  tests  with  level  and  power 
completely  independent  of  the  unknown  variances. 

The  next  section  briefly  outlines  the  two-way  layout 
heteroscedastic-ANOVA  (HANOVA)  methodology.  It  is  followed  by  case  study 


■  ...  -...■■•ei. .......... 


firra  simulation  for  the  two-way  layout 


of  Lin's  [1978]  multiple  objective 
ANOVA  procedure.2  Finally,  the  conclusion  and 
applications  are  outlined. 


other  possible  business 


THE  HANOVA  PROCEDURE 


Bishop  and  Dudewicz  [1978]  [1981]  have  developed  heteroscedastic 
ANOVA  (HANOVA)  procedures  in  an  r-way  layout.  For  the  purpose  of  this 
paper,  only  the  two-way  layout  will  be  described. 

The  two-way  layout  under  consideration  is  what  is  usually  studied  in 
ANOVA  and  is  defined  by 


X.  =  p  +  a. 
ijk  M  1 


p.  +  ap.  .  -t-  e. 


(i  =  1,  ....  I;  j  =  1,  ....  J;  k  =  1,  2 . N) , 


where  (X.  ..  ]  are  the  observed  responses;  it  is  assumed  that  the  [X.  ]  are 

ljk  r  ljk 

independent  and  normally  distributed  with  unknown  mean  E(X  .  .,  )  =  (j  and  un- 

i  j  k 

2 

known  variance  Var(X^^)  =  ;  [e^^l  are  assumed  to  be  independent  ran- 

2 

dom  variables  with  normal  distributions  with  mean  0  and  variance  o. .  ,  denoted 

ij 

2  2 

e.  ..  ~N(0,  a .  .  ),  0  <  a.  .  <  |j  is  the  overall  mean;  a .  and  p .  are 

tj  k  ij  ij  i  j 

the  main  effects  of  factors  i  and  j;  0(8. .  is  the  interaction  between  i 

ij 

and  j ;  and  that 


1  a.  =  I  p.  =  1  a8..=  I 
i-1  1  j=l  J  i=l  ,J  j=l 


ap.  .  =  0. 
ij 


The  hypotheses  of  interest  are 

Hq :  Cl  =  0  for  all  i  (2 

H.:  p.  =  0  for  all  j  (• 

1  j  J 

H„:  op..  =0  for  all  i  and  j  (4 

2  rij  J 

In  this  two-way  layout  there  are  I  x  J  possible  treatment 
combinations.  Cell  (i,  j)  refers  to  the  combination  of  level  l  of  the 


first  factor  and  level  j  of  the  second  factor.  One  seeks  tests  of  the 
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above  three  hypotheses  based  on  test  statistics  whose  distributions  are 
independent  of  the  unknown  variances. 

Bishop  and  Dudewicz  [1978]  [1981]  developed  the  following  HANOVA 

procedure  to  test  the  above  hypotheses  based  on  test  statistics  whose 
distributions  are  independent  of  the  unknown  variances  and  both  the  level 
(i.e.,  probability  of  Type  I  error)  and  power  (i.e.,  probability  of 
Type  II  error)  of  the  test  are  controllable: 

1.  Choose  the  Type  I  error  level,  Type  II  error  level,  and  the  tol¬ 
erance  of  the  difference  between  populations. 

2.  Compute  the  design  constant  z  >  0  given  the  parameters  specified  in 


Step  1,  and  then  in  each  cell  (i,j)  take  an  initial  sample  X. 


ijl’ 


( .  X ......  X. .  ,  of  size  n  from  each  of  k  populations. 

ij2  ij3  ljn  o 


Compute  sample  variance  s„  ,  the  usual  unbiased  estimate  of  unknown 

2 

population  variances  ,  and  define  the  final  sample  size: 

2 

s .  . 

N.  .  =  max[n  +  1,  +  1} 

n  o  z 

where  [X]  denotes  the  largest  integer  which  is  smaller  than  X. 


Take  (N .  .  -  n  )  additional  observations  from  cell  (i,j)  populations 
ij  ° 


and  calculate  the  weighting  coefficients  a^.j, 

1 


3ijl  ' 
aijn0^ 


=  a .  . 
tin 

c 

.  .  .  =  a 


ijN 


ij 


(N.  .  -  n  )  b.  . 
ij  o  ij 


b.  . 
J-J 


a  .  ..  such  that 

ijN .  . 


and  b  .  .  =  rr^— 
ii  N  .  . 
IJ 


1  + 


n  (N.  .  z  -  s 
°  ij _ y__ 

■1  (N.  .  -  n  )  s .  ,2 

O 
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5.  Compute  ceLl  sample  generalized  means  X. 


ij  ■ 


ij 

1  a 
k- 1 


ljk  Xijk 


where  X,.,,  .  .  .  ,  X .  ...  is  the  finai  set  of  observations  for  cell 
ij 1  ijN  .  . 

ij 

Then  compute  group  means  for  treatments  i  and  j: 

=  !  J  . 

X  =4  2  X . 

1..  JJ=1  IJ. 


*  1  1  * 

x  =4  i  x. , 

•J.  I  i=1  IJ. 


and  grand  mean: 

*  !  *  J  * 

X. . .  =  =4  I  2  X  . 

IJ  •  ,  ij  • 

i=l  j=l 

6.  Test  hypotheses  (2),  (3) ,(4)  based  on  the  following  quadratic  forms 
of  independent,  identically  distributed  Student's  t  variates: 


i  (x.  -  x...r 

F  =  J  I  — ^ - 

0  i=l 


15) 


J  (X  .  -  x...)- 


r> =  1 A 


(6) 


I  J  (X. .  -  X  -  X  .  +  X...) 


F  =  1  l 

i=l  j=l 


LL  -J- 


17) 


The  test  of  proceeds  by  rejecting  H  if  and  only  if  Fq  >  ^ 

’  o 

where  F,a  is  the  upper  a1-^  percent  point  of  the  null  distribution  of 
’  o 

Fq.  Tests  for  the  other  hypotheses  are  performed  in  a  similar  manner, 

namely  reject  (or  H^)  if  and  only  if  the  corresponding  statistic 

(or  F 2)  is  greater  than  the  upper  percent  point  F (or  F ^ )  of 

•no  ,no 

its  respective  null  distribution.^ 

In  general,  this  new  procedure  uses  z  to  replace  traditional  F-test's 

2 

The  power  of  the  test  is  one  of  the  inputs  to  determine  the  value 
of  z.  Therefore,  the  Type  II  error  can  be  controlled.  The  new  procedure 


also  uses  generalized  ceil  means  to  replace  the  traditional  F-test 

cell  sample  means.  Bishop  and  Dudewicz  [1981]  have  proved  that  (X.^  1 

~  ~  ~  2 
are  independent  random  variables  and  F^,  F ^ ,  F^  are  independent  of  0^  . 

Therefore,  the  assumption  of  equal  variances  can  be  eliminated  by  using 

the  new  procedure.  Bishop  and  Dudewicz  [1981]  also  showed  that  the  power 

of  the  test  using  ?  is  much  better  than  the  power  of  the  traditional 


F-test  when  the  variances  are  unequal.  Therefore,  the  new  procedure  is 
more  efficient  than  the  traditional  procedure. 


MULTIPLE  OBJECTIVE  BUDGETING  SIMULATION  CASE  STUDY 


Most  of  the  business  decisions  faced  by  managers  and  accountants  are 
multiple  criteria  or  objective  decision  problems.  Lin  [1978]  conducted  a 
simulation  study  of  a  hypothetical  firm  with  multiple  objectives  of  profit 
and  sales.  He  used  the  traditional  F-tests  to  examine  the  effects  of 
alternative  planning  models  and  accounting  variance  analysis  techniques 
on  the  firm's  profit  and  sales  performance.  The  simulation  results  showed 
that  profit  and. sales  under  multiple  objective  linear  programming  are 
higher  than  those  under  goal  programming  planning  model.  When  comparing 
ex  post  accounting  variance  analysis  with  traditional  variance  analysis, 
the  former  resulted  in  higher  sales,  but  there  was  no  significant  dif¬ 
ference  in  profits.  The  major  reason  was  that  the  observed  sample  means 
were  very  close:  hence,  the  power  of  the  test  was  very  low.  Since  the 
variances  were  unknown,  the  traditional  F-test  was  unable  to  set  its 
sample  size  to  control  the  power.  We  now  apply  the  new  HANOVA  test.b 

The  main  objective  of  the  simulation  experiment  is  to  study  the 
effects  of  combinations  of  production  planning  models  and  accounting 
variance  analysis  techniques  on  profit  and  sales  performances.  This  study 
uses  a  2  x  2  factorial  design  as  shown  in  Figure  1. 


Variance  Analysis  Factor 


Planning 

Mode  1 

Mode  2 

Factor 

Mode  3 

Mode  4 

Figure  1,  2“  Factorial  Experiment 
The  four  treatment  combinations  are: 

Mode  1  Using  goal  programming  planning  model  and  traditional 
variance  analysis  technique  (GP-TRAD) 
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Mode  2  Using  goal  programming  planning  model  and  ex  post 

variance  analysis  technique  (GP-EXPO) 

Mode  3  Using  multiple  objective  linear  programming  planning 

model  and  traditional  variance  analysis  technique  (MOLP- 
TRAD) 

Mode  4  Using  multiple  objective  linear  programming  planning 

model  and  ex  post  variance  analysis  technique  (MOLP-RXPO) 

The  overall  model  is  a  production  planning  decision.  The  planning 
period  is  assumed  to  be  a  month.  The  total  run  length  for  the  short-run 
production  decisions  is  assumed  to  be  two  years  or  twenty-four  periods. 
Sample  size  is  determined  by  replicating  runs  of  this  total  run  length 
using  different  sets  of  pseudo-random  numbers. 

Following  Bishop  and  Dudewicz's  [1981]  recommended  approximation 
procedure  for  the  two-way  layout,  the  limiting  distribution  of  FQ  is  non- 
central  chi-square  with  one  degree  of  freedom  and  noncentrality  parameter 


K0 


2  ‘  .1.  a 

i=l  i 


,  denoted  by  X^(AQ).  Similarly,  it  can  be  shown  that  the 


limiting  distribution  of  is  noncentra  1  chi-square  with  one  degree  of 


freedom  aad  noacentrality  parameter  A 


2  ’  .5  p‘ 
_  1=1  .  I 


,  while  the  limiting 


distribution  of  F0  is  noncentral  chi-square  with  one  degree  of  freedom 


ig1  |  (op..)' 

and  noncentrality  parameter  A,,  =  — — ^ - ** — 

2  z 
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For  example,  in  this  simulation  the  level  of  the  Lest,  i.e., 
probability  of  Type  I  error,  was  assumed  to  be  0.05,  with  power  at  least 
0.95,  i.e.,  Type  II  error  probability  =  0.05,  and  the  tolerance  of 

difference  between  modes  for  profit  and  sales  were  assumed  to  be  $30  and 
$100,  respectively. 1  From  Haynan,  Govindarajulu  and  Leone's  [1970] 
tables  of  the  cumulative  noncentral  chi-square  distribution,  the 
noncentrality  parameters  are  12.995  for  both  profit  and  sales.  Then  the 
standard  errors  of  estimate  z  for  profit  and  sales  are  69.2574  and 
769.5267  respectively. 

Taking  initial  sample  size  of  n  =  30,  the  sample  means  and  variances 

g 

are  shown  in  Table  1. 

Insert  Table  1  Here 

The  final  sample  sizes  and  other  simulation  statistics  are  shown  in 
Table  2. 9 

Insert  Table  2  Here 

2 

From  the  central  chi-square  table,  the  critical  value  for  is 

3.84.  From  the  simulation  outputs,  the  actual  profit  under  goal 
programming  is  significantly  different  from  the  profit  under  multiple 
objective  linear  programming  planning  models,  since  FQ  =  3.99.  Lin's 
[1978]  study  also  showed  a  significant  difference  of  profit  between  two 
models.  The  actual  sales  under  goal  programming  is  also  significantly 
different  from  the  sales  under  multiple  objective  linear  programming 
planning  models  with  F^  =  75.40.  This  result  is  consistent  with  Lin's 
study.  When  comparing  ex  post  variance  analysis  with  traditional  variance 
analysis,  Lin  [1978]  found  that  the  former  results  in  statistically  higher 
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means  of  actual  sales  per  period,  but  that  there  is  no  significant 
difference  on  the  means  of  actual  profit  per  period.  One  major  reason  for 
nonsignificance  in  profit  was  that  the  power  of  the  test  ;  .  his  study  was 
low.  In  this  study,  the  power  of  test  is  95%,  the  siw  ’  d  results  (F^  = 

4.85  for  profit  and  5.54  for  sales)  show  that  there  are  a.-stlstically 

significant  differences  on  both  profit  and  sales  p  rtontiance  between 
traditional  and  ex  post  accounting  variance  analysis  techniques. 

In  general,  Bishop  and  Dudewicz's  [1978]  [1981]  HANOVA  procedures 
eliminate  the  necessity  of  the  assumption  of  equal  variances.  In 
addition,  the  experimenter  receives  the  added  benefits  of  a  controllable 
power  function  and  allocation  of  additional  sample  size  where  it  is 
important.  The  new  simulation  results  show  that  multiple  objective  linear 
programming  results  in  higher  profit  and  sales  than  those  under  goal 

programming  planning  models.  This  is  consistent  with  the  theory  that  the 

former  is  an  optimizing  model  while  the  latter  is  a  satisficing  model, 
when  comparing  ex  post  variance  analysis  with  traditional  accounting 
variance  analysis,  the  forater  results  in  higher  amounts  of  both  profit  and 
sales.  This  result  is  consistent  with  the  theory  that  the  ex  post 
variance  analysis  uses  more  relevant  feedback  information  if  the 
information  is  costless. 
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CONCLUSION 


The  assumption  of  equal  variances  in  ANOVA  is  often  in  doubt, 
especially  when  most  business  and  accounting  variables  do  not  come  from 
equal  variance  populations.  Previous  accounting  or  business  simulation 
studies  such  as  Deraski's  [1967]  [1970]  implementation  effects,  Sundera's 
[1974]  evaluating  capital  budgeting  models,  Onsi's  [1975]  simulation  of 
organizational  slack,  Fellingham,  Mock,  tfasarhelyi ’ s  [1976]  information 
choice,  and  Magee's  [1976]  analysis  of  alternative  cost  variance  models 
have  assumed  equal  variances  and  encountered  inability  to  control  Type  II 
error.  Lin's  [1978]  simulation  of  multiple  objective  budgeting  models 
failed  to  control  Type  II  error.  Unfortunately,  this  means  that  a  failure 
to  assert  a  model  effect  in  a  simulation  of  accounting  or  business  systems 
may  be  due  not  to  the  nonexistence  of  such  effects,  but  rather  to  a 
Type  II  error. 

This  paper  introduces  a  new  methodology  which  eliminates  the 
necessity  of  such  an  assumption  of  equal  variances.  In  addition,  the 
experimenter  or  manager  receives  the  added  benefits  of  a  controllable 
power  function  and  allocation  of  the  additional  sample  size  "where  it 
counts"  (i.e.,  where  variability  is  high  and  may  obscure  means).  An 
extension  of  Lin's  [1978]  multiple  objective  budgeting  simulation  has 
been  illustrated  as  a  case  study.  The  new  HANOVA  procedures  can  be 
applied  to  many  other  business  simulation  experiments.  For  example,  the 
different  inventory  costing  methods  can  be  considered  together  with 
different  depreciation  methods  to  test  the  effects  of  different  factors  on 
profit  performance.  Other  applications  include  different  audit 
procedures  in  variable  sampling,  different  job  shop  scheduling  schemes, 
and  different  advertising  strategies. 
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FOOTNOTES 


Type  I  error  is  defined  as  the  error  of  rejecting  the  null  hypothesis 
when  it  is,  in  fact,  true.  Type  II  error  is  defined  as  the  error  of 
ar.cc;  wing  the  null  hypothesis  when  it  is  actually  false.  Experiments 
in  situations  where  variances  are  unknown  and  probably  unequal  are 
called  "heteroscedastic  experiments."  Kleijnen  and  Naylor  [1969] 
observed  that  "Only  in  rare  cases  can  the  assumption  of  a  common 
known  variance  be  expected  to  hold  with  computer  simulation 
experiments  with  models  of  business  and  economic  systems"  (pp.  609- 
610) . 

The  reason  to  replicate  Lin's  multiple  objective  budgeting 

simulation  model  is  that  it  is  a  significant  improvement  over 

previous  accounting  simulations  in  terms  of  external  validity.  It  is 

a  more  sophisticated  model  with  an  improved  analytical  technique. 

One  may  view  z  as  playing  the  role  of  o“/N,  standard  error  of  the 

mean,  in  the  traditional  r-way  layout  when  the  errors  ha v  ■*  al 
2  2 

variance  a  ,  a  is  known,  and  N  observations  are  taken  in  each  cell. 
Bishop  and  Dudewicz  [1978]  developed  tables  for  choosing  z  vaLues  in 
the  one-way  layout.  For  the  two-way  layout  z  values  are  related  to 
noncentrality  parameters  in  noncentral  chi-square  distributions. 

While  other  choices  of  the  {a„  }  are  possible,  the  above  are  con¬ 
jectured  by  Dudewicz  and  Dalai  [1975]  to  be  robust  against 
nonnormality  of  the  errors. 

The  cutoff  points  (F_a  ),  (F,a  ),  and  (F.,a  )  are  not  tabulated 

r  0,nl,n  2 ,  n 

o  ’  o  o 

in  the  literature  yet,  and  thus  an  approximation  is  called  for.  It  has 
been  shown  by  Bishop  and  Dudewicz  [1981]  that  the  limiting  distribution 
of  Fq  is  noncentral  chi-square  with  1-1  degrees  of  freedom  and  non- 
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centrality  parameter  Aq  =  J  I  ii‘  /  z,  denoted  by  \t _  j  (Aq) . 

i  =  l 

Note  that  the  closeness  of  the  approximation  depends  in  no  way  upon 
2  ~ 

the  •  That  is,  Fg ,  F,  ,  and  F0  are  independent  of  unknown 
population  variances. 

6.  See  Lin  [1978]  for  the  description  of  the  overall  model  in  terms  of 
planning,  operations,  and  performance  evaluation  processes.  Lin 
[1980]  also  showed  examples  of  ex  post  accounting  variance  analysis 
under  both  goal  programming  and  multiple  objective  linear  program¬ 
ming  planning  models. 

o  2 

2  2 

7.  That  is,  I  a'T  and  2  8  .  are  each  assumed  to  be  530  for  profit, 

i=l  1  J=1  J 

and  each  $100  for  sales. 

8.  The  initial  sample  size  should  be  large  enough  to  allow  Central  Limit 
Theorem-type  effects  to  take  hold,  and  large  enough  to  keep  the 
multiplier  z,  used  to  set  the  second  stage  sample  sizes,  relatively 
small.  A  number  of  around  20  seems  desirable,  while  a  number  less 
than  10  is  not  recommended. 

9.  In  Lin's  [1978]  study,  the  sample  size  of  the  profit  variable  for  all 
four  modes  was  126.  The  power  of  that  test  was  around  0.80. 
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Table  1 


INITIAL  SAMPLE  MEANS  AND  VARIANCES 


(no  =  30) 


Variables 


Mode  1 


Mode  2 


Mode  3 


Mode  4 


,  (GP-TRAD)  iCGP-EXPO)  (MOLP-TRAD)  .  (MOLP-EXPO) 


Actual  Profit  Mean  762 

Per  Period:  Variance  12309 


792 
j  11794 

I 

I 

i 


812 

10078 


812 

i0096 


Actual  Sales  Mean  !  6220  6364  1  6539  6537 

Per  Period:  Variance  38682  33591  39816  37998 
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Table  2 


SIMULATION  STATISTICS 


Variables  and  Modes 


Final 

r 

|  Weighting 

i  Weighting 

1  .  r.  .  ... - P- 

:  i 

; Generalized  i 

Sample 

1 

i  Coefficient 

f 

i  Coefficient 

' Cell  Mean 

Other 

Size 

a .  . 

i  b  .  . 

i  'v 

i 

Statistics 

j  J-J 

1  ij 

1 

I  X 

1 

I 

i  * 

Profit: 


Mode 

1: 

GP-TRAD 

178 

.0052 

.0057 

762 

X.  . 

.=  788 

Mode 

2: 

GP-EXPO 

171 

.0051 

.0060 

796 

f0 

=  3-99 

Mode 

3: 

MOLP-TRAD : 

146 

.0067 

.0069 

|  794 

=  4.85 

Mode 

4: 

MOLP-EXPO 

146 

■  .0063 

.0070 

797 

?2 

=  3.37 

Sales  : 


Mode 

1 : 

GP-TRAD 

51 

.0177 

Mode 

2: 

GP-EXPO 

44 

.0213 

Mode 

3: 

MOLP-TRAD 

52 

.0181 

Mode 

4; 

MOLP-EXPO 

50 

.0182 

0244 

6204 

X.  . 

.  =  6388 

0257 

6330 

=  75.40 

0208 

6506 

=  5.54 

0227 

6510 

F2 

=  4.84 
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